Rewards Analytics and Distribution Dashboard

This goal of this notebook is to offer an easy way to process the outputs of the praise and sourcecred reward systems, perform an analysis of the results and calculate the token reward distribution. It uses mock data and should be considered a work-in-progress.

Imports

First, we import the relevant libraries, get the Data and set how many tokens we want to distribute

Now that we have selected the files, we can import them for processing. We can also set the total amount of tokens we want to distribute this period. We also set the name we want our output file to have.

Tip: Now that the file paths are set, you can safely click on "Cell > Run all below" from here on the menu bar to execute everything :)

Parameters

Set the total amount of tokens we want to distribute this period. We also set the name we want our output file to have.

Results Analysis

Let's dive into some data analysis! We'll use the metrics designed and explained by octopus🐙, and focus only on the praise rewards for now. Starting with:

Allocation percentages - Praise receivers

This table will show us which percentage of the total rewards gets distributed to which top % of users. So "Top 50% -> 0.85" would mean that the top 50% of praisees received 85% of the total rewards

	Rewards
Top 50%	0.970013
Top 20%	0.836119
Top 10%	0.611844
Top 5%	0.376984
Top 1%	0.146073

Allocation Percentages - Praise givers

This table will show us which percentage of the total rewards get distributed based on praise given by which top % of users. So "Top 50% -> 0.85" would mean that 85% of the total rewards are based on praise given by the top 50% praise givers.

	Rewards
Top 50%	0.969677
Top 20%	0.851743
Top 10%	0.753717
Top 5%	0.640571
Top 1%	0.367073

Plotting praise given against praise received

Now we can compare the relationship between praise given by a user and the praise they received

Gini coefficient

Next we will look at the Gini coefficient. Note that there is some debate if we want to use this metric at all, since it is usually employed to measure wealth distribution, and not compensation.

	Rewards
All	0.766340
Top 50%	0.604118
Top 20%	0.320445

Shannon Entropy

Shannon Entropy) is a concept from communications theory, which is also used in measuring the diversity of a distribution. The formula for calculating Shannon Entropy among $n$ individuals is $$\\sum_{k=1}^n -p_k log_2(p_k),$$ where $p_k$ represents the proportion of the resource that user $k$ received.

Here we compare the actual Shannon Entropy with the maximum possible for the dataset, keeping in mind that a Shannon Entropy of 0 would mean one user holds all the rewards

	Rewards
Entropy	5.434770
Max Entropy	7.118941
% of Max	0.763424

Nakamoto Coefficient

Last but not least, the Nakamoto coefficient. The Nakamato Coefficient is defined as the smallest number of accounts who control at least 50% of the resource. Although its significance relates to the prospect of a 51% attack on a network, which may not be relevant in our context, we can still use it as an intuitive measure of how many individuals received the majority of a resource.

11

Praise Data Visualization

Rating distribution

Since praise gets valued on a scale, we can take a look at how often each value of the scale gets assigned by quantifiers.

Praise Reward Distribution

We can also take a look at the distribution of the received praise rewards

Praise Outliers

To aid the revision process, we can generate a table which sorts the praise by the size of the spread between the highest and lowest scores. These "controversial" praise instances can then be discussed in the quantifier review call. Index numbers are kept to make localization in the "big praise sheet" easier.

	index	ID	DATE	TO USER ACCOUNT	FROM USER ACCOUNT	REASON	SCORE 1	SCORE 2	SCORE 3	DUPLICATE ID 1	DUPLICATE ID 2	DUPLICATE ID 3	DISMISSED 1	DISMISSED 2	DISMISSED 3	QUANTIFIER 1 USERNAME	QUANTIFIER 2 USERNAME	QUANTIFIER 3 USERNAME	AVG SCORE	SPREAD
0	724	621f79e243b89009366c8fd5	2021-09-17T22:00:00.000Z	Suga#8514	Mount Manu#3530	for creating the Communitas WG 🏘️	144	0	2	None	621f79e243b89009366c8fcb	621f79e243b89009366c8ff2	False	False	False	Mount Manu#3530	liviade#1387	Parrachia#0450	50	144
1	26	621f79e243b89009366c8ceb	2021-09-01T22:00:00.000Z	elessar.eth#7945	griff (💜, 💜)#8888	for EVM Crispr, an amazing tool	144	0	3	None	None	None	False	True	False	Zeptimus#3359	Mettodo#7010	acidlazzer#5796	50	144
2	47	621f79e243b89009366c8d00	2021-09-02T22:00:00.000Z	iviangita#3204	iviangita#3204	for all the incredible behind-the-scenes work ...	0	3	144	None	None	None	True	False	False	innov8tor3#3988	enti#1546	Maxwe11#7157	56	144
3	567	621f79e243b89009366c8f21	2021-09-13T22:00:00.000Z	natesuits#4789	Suga#8514	for creating TEC Source! https://app.gitbook.c...	144	21	1	None	None	None	False	False	False	Mount Manu#3530	Mettodo#7010	bends#3537	44	143
4	368	621f79e243b89009366c8e4c	2021-09-09T22:00:00.000Z	liviade#1387	Mount Manu#3530	for all the SoftGov work	1	1	144	None	None	None	False	False	False	PhilH#1102	enti#1546	Parrachia#0450	37	143
5	873	621f79e243b89009366c906a	2021-09-18T22:00:00.000Z	natesuits#4789	Mount Manu#3530	for TEC Source <:TEC:835016327542210560>	144	21	1	None	None	None	False	False	False	Mount Manu#3530	Mettodo#7010	bends#3537	41	143
6	171	621f79e243b89009366c8d87	2021-09-02T22:00:00.000Z	fabiomendes#4802	Mount Manu#3530	for working all day with Pedro on the dashboard	144	2	2	None	621f79e243b89009366c8d80	621f79e243b89009366c8d80	False	False	False	Mount Manu#3530	AmwFund#0979	Maxwe11#7157	73	142
7	3	621f79e243b89009366c8ccf	2021-08-31T22:00:00.000Z	CurlyBracketEffect#5572	chuygarcia.eth#6692	for launching Agave <:AlvinBright:842140083179...	3	3	144	None	None	None	False	False	False	Zeptimus#3359	AmwFund#0979	Maxwe11#7157	42	141
8	381	621f79e243b89009366c8e59	2021-09-09T22:00:00.000Z	Nuggan#5183	Mount Manu#3530	for making the CCD beautiful 🦄	3	5	144	None	None	None	False	False	False	PhilH#1102	Vyvy-vi#5040	Maxwe11#7157	43	141
9	697	621f79e243b89009366c8fba	2021-09-17T22:00:00.000Z	kristofer#1475	Mount Manu#3530	for his leadership in both the rewards channel...	34	3	144	None	None	None	False	False	False	innov8tor3#3988	enti#1546	Parrachia#0450	48	141
10	1422	621f79e243b89009366c92f6	2021-09-29T22:00:00.000Z	Vitor Marthendal#0090	divine_comedian#5493	for their magic on the CCD	3	144	21	621f79e243b89009366c8fbb	None	None	False	False	False	lee0007#8152	Vyvy-vi#5040	teaci#6108	45	141
11	1401	621f79e243b89009366c92e1	2021-09-29T22:00:00.000Z	natesuits#4789	Juankbell#7458	for being the 3rd and 4rd Graviton training fa...	144	5	8	None	None	None	False	False	False	Mount Manu#3530	Mettodo#7010	bends#3537	44	139
12	326	621f79e243b89009366c8e22	2021-09-08T22:00:00.000Z	iviangita#3204	Zeptimus#3359	for finding out our twitter account as block a...	21	5	144	None	None	None	False	False	False	innov8tor3#3988	enti#1546	Maxwe11#7157	47	139
13	862	621f79e243b89009366c905f	2021-09-18T22:00:00.000Z	natesuits#4789	Mount Manu#3530	for their support in the creation of the Commu...	144	5	13	None	None	None	False	False	False	Mount Manu#3530	Mettodo#7010	bends#3537	41	139
14	1120	621f79e243b89009366c91b9	2021-09-23T22:00:00.000Z	elessar.eth#7945	iviangita#3204	for the amazing work on the smart contracts fo...	144	5	13	None	None	None	False	False	False	Zeptimus#3359	Mettodo#7010	acidlazzer#5796	41	139
15	534	621f79e243b89009366c8efe	2021-09-11T22:00:00.000Z	r33pich33p#6906	griff (💜, 💜)#8888	for hacking on the Sunday Funday Params Party ...	144	21	5	None	None	None	False	False	False	Mount Manu#3530	Vyvy-vi#5040	teaci#6108	43	139
16	533	621f79e243b89009366c8efd	2021-09-11T22:00:00.000Z	natesuits#4789	griff (💜, 💜)#8888	for hacking on the Sunday Funday Params Party ...	144	8	13	None	None	None	False	False	False	Mount Manu#3530	Mettodo#7010	bends#3537	43	136
17	562	621f79e243b89009366c8f1c	2021-09-13T22:00:00.000Z	natesuits#4789	Tam2140#9361	for the herculean effort that must have gone i...	144	21	34	None	None	None	False	False	False	Mount Manu#3530	Mettodo#7010	bends#3537	55	123
18	24	621f79e243b89009366c8ce9	2021-09-01T22:00:00.000Z	fabiomendes#4802	divine_comedian#5493	for their brazilian hacking skills on the CCD	34	21	144	None	None	None	False	False	False	Mount Manu#3530	AmwFund#0979	Maxwe11#7157	51	123
19	379	621f79e243b89009366c8e57	2021-09-09T22:00:00.000Z	divine_comedian#5493	Mount Manu#3530	for making the CCD beautiful 🦄	21	21	144	None	None	None	False	False	False	innov8tor3#3988	liviade#1387	Maxwe11#7157	51	123
20	342	621f79e243b89009366c8e32	2021-09-09T22:00:00.000Z	divine_comedian#5493	Mount Manu#3530	for their work in the Params WG 🤖	5	0	89	None	621f79e243b89009366c8d75	None	False	False	False	innov8tor3#3988	liviade#1387	Maxwe11#7157	28	89
21	36	621f79e243b89009366c8cf5	2021-09-01T22:00:00.000Z	divine_comedian#5493	divine_comedian#5493	for design, development, ideation and copywrit...	0	34	89	None	None	None	True	False	False	innov8tor3#3988	liviade#1387	Maxwe11#7157	52	89
22	953	621f79e243b89009366c9108	2021-09-21T22:00:00.000Z	iviangita#3204	Zeptimus#3359	for drafting the first transparency budget	89	3	0	None	None	None	False	False	True	innov8tor3#3988	enti#1546	Maxwe11#7157	37	89
23	25	621f79e243b89009366c8cea	2021-09-01T22:00:00.000Z	pKrepe#5682	divine_comedian#5493	for their brazilian hacking skills on the CCD	55	0	89	None	None	None	False	True	False	PhilH#1102	Mettodo#7010	bends#3537	52	89
24	752	621f79e243b89009366c8ff1	2021-09-17T22:00:00.000Z	Nuggan#5183	Mount Manu#3530	for his work in Params 👨🏻‍💻	0	21	89	621f79e243b89009366c917a	None	None	False	False	False	PhilH#1102	Vyvy-vi#5040	Maxwe11#7157	41	89

We can also plot the spread against the average score for a more visual search.
ATTENTION! Keep in mind that this is just meant as a visual aid. If there are several praise instances with similar spread and average quant number, all but one end up "hidden" on the chart. For the full list you have to use the table.

Praise Flows

Now for something more fun: let's surface the top "praise flows" from the data. Thanks to @inventandchill for this awesome visualization! On one side we have the top 20 praise givers separately (modifiable by changing the variable n_senders), on the other the top 25 receivers (modifiable by changing the variable n_receivers). The people outside the selection get aggregated into the "REST FROM" and "REST TO" categories.

SourceCred Data

SourceCred token Distribution

Next we can see the distribution made by the SourceCred algorithm.

Quantifier Data

Let's take a closer look at each quantifier. In the following step we will use the raw praise data to zoom in on how each quantifier scored the praises:

Amount of praise quantified

With the above table we can easily see how much praise each quantifier rated.

Mean praise score by quantifier

Let's see the mean praise scores every quantifier gave.

Total Token Distribution Visualization and Export

To send the allocations to the DAO for distribution, we need to put all data together and add the rewards for the reward board and the quantifiers.

Let's take a final look at the total distribution. Click on the legend to filter out specific reward sources

That's it! We can now download the final distribution files and upload them to GitHub for future reference. We want to export 3 different files:
 - The final reward allocations, separated by source
 - The final reward allocations, in a disperse.app-readable format
 - The extended praise data, detailing the how many tokens each single praise netted (for future reference)